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ABSTRACT 

Experimental  results  for  the  recognition  of  general  curves  in  three  space  using 
registered  range  and  intensity  images  are  presented.  The  matching  algorithm  uses 
fast  Fourier  transforms  to  determine  the  least  squares  difference  between  sequences 
of  points  sampled  at  equal  intervals  along  two  piecewise  linear  approximations  of 
curves  in  three-space  and  returns  the  rotation  and  translation  required  to  bring  one 
of  the  curves  into  closest  juxtaposition  with  the  other.  Performance  of  the  algorithm 
is  demonstrated  by  matching  curves  in  three  space  which  are  the  boundaries  of 
regions  of  contrasting  reflectivity  on  curved  surfaces.  The  experiments  use  a 
recently  developed  range  sensor  which  is  able  to  generate  a  512x460x12  bit  range 
image  (with  registered  intensity  image)  in  40  seconds. 


1.   Introduction 

General  curves  in  three-space  may  be  extremely  useful  as  features  for  object 
recognition  because  curves  relevant  to  object  shape  can  be  extracted  using  low  level 
image  processing  techniques  and  can  be  easily  stored  and  manipulated.  Examples 
of  curves  might  be  the  edges  of  a  pottery  shard,  the  boundaries  of  a  painted  design, 
a  line  of  locally  maximum  curvature  (as  at  a  cheekbone),  or  the  intersection  of  a 
wall  with  a  ceiling.  The  significance  of  such  curves  is  suggested  by  the  human  abiHty 
to  identify  a  complex  scene  from  relatively  few  lines  in  cartoon  sketches.  While  the 
full  extent  to  which  curves  in  three-space  can  be  used  for  object  recognition  remains 
to  be  determined,  preliminary  experiments  presented  in  this  paper  demonstrate  that 
3-D  objects  can  be  located  and  identified  on  the  basis  of  curves  extracted  from 
registered  range  and  intensity  images. 

The  matching  algorithm  used,  described  previously  in  [SchSh85],  compares  two 
3-D  curves,  one  of  which  is  an  observed  curve  to  be  identified  and  the  other  a 
known  curve,  or  model,  and  calculates  the  position  and  orientation  of  the  model 
curve  that  best  matches  the  observed  curve.  A  least  squares  difference  value  is  also 
computed.  When  identifying  an  observed  curve  as  one  of  a  set  of  models,  the 
model  with  the  best  least  squares  difference  is  selected.  Since  the  inputs  are 
piecewise  linear  curves  with  an  arbitrary  number  of  pieces,  the  algorithm  will  work 
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for  any  digitized  curves.  Additionally,  the  observed  curve  may  be  a  small  subsection 
of  a  model  curve,  so  that  it  is  possible  to  identify  partially  occluded  curves.  The 
performance  of  this  algorithm  in  actual  practice  depends  critically  on  our  ability  to 
obtain  accurate  three-dimensional  data,  for  which  we  rely  on  range  sensing. 

The  range  sensor  described  in  section  4  uses  a  structured  beam  of  light  and  a 
video  camera  to  record  intensity  values  from  which  range  values  are  calculated  on 
an  image  processing  system.  The  output  of  the  sensor  is  a  set  of  registered  range 
and  intensity  images  of  the  work  scene,  which  may  include  solid  objects.  While  an 
'intensity  image'  is  conventionally  understood  to  be  a  rectangular  array  of  values 
which  represent  the  intensity  of  light,  e.g.  as  recorded  by  digitizing  the  output  of  a 
video  camera,  the  term  'range  image'  has  been  used  to  mean  several  things.  We 
use  'range  image'  to  denote  a  rectangular  array  of  values  which  represent  distance 
from  a  plane  perpendicular  to  the  camera  axis.  A  range  image  is  said  to  be  in 
registration  with  an  intensity  image  when  a  patch  of  any  surface  present  in  the  work 
area  is  projected  onto  corresponding  elements  of  both  images. 

Before  describing  how  significant  edges  might  be  extracted  from  registered 
range/intensity  images,  we  comment  on  some  advantages  provided  by  this  approach. 
In  [L85],  it  is  argued  that  solid  bodies  are  identifiable  from  lines  extracted  from 
simple  projections.  The  line  drawing  of  Fig.  1.1,  for  example,  is  instantly 
recognized  by  the  reader,  and  there  is  evidence  that  lines  play  a  fundamental  role  in 
human  visual  processing  [HB62].  However,  a  line  drawing  does  not  specify  the  size 
of  an  object,  and  shapes  are  ambiguous:  Fig.  1.1  could  represent  an  elliptical  flower 
pot  instead  of  a  more  common  circular  one.  The  lines  in  Fig.  1.1  represent  a 
projection  of  curves  which  exist  in  three-space,  and  it  is  these  curves  which 
determine  the  size  and  shape  of  the  object  unambiguously.  Curves  such  as  these  on 
the  surface  of  a  solid  object  can  be  described  using  points  appropriately  extracted 
from  a  range  image  of  the  object,  as  indicated  in  the  next  paragraph. 

First,  we  note  that  the  lines  of  Fig.  1.1  all  represent  sharp  intensity  gradients, 
caused  by  such  features  as  changes  of  reflectivity,  the  edge  of  an  object,  or  sharp 
changes  of  surface  curvature,  and  are  intuitively  significant  for  the  recognition  of 
the  shape  of  the  objects  (the  pot,  the  decal).  (We  assume  ordinary  diffuse  lighting 
here,  since  illumination  can  be  contrived  to  make  an  object  blend  with  its 
background,  or  even  to  make  sharp  curvature  changes  disappear,  as  in  the  case  of 
the  center  of  a  fluted  column  which  is  viewed  with  a  point  light  source  behind  and 
above  the  viewer.)  Suppose  that  registered  intensity  and  range  images  of  the  pot 
have  been  made.  Then  points  on  the  curves  represented  by  the  lines  of  Fig.  1.1  can 
be  found  by  applying  edge  detectors  to  the  intensity  image  and  using  the 
corresponding  pixels  of  the  range  image. 

In  addition  to  curves  easily  found  in  intensity  images,  curves  of  geometric 
significance  such  as  lines  of  maximum  local  curvature  or  lines  separating  regions  of 
different  curvature  (e.g.  the  smooth  join  of  a  cylinder  to  a  plane  tangent  to  it)  could 
be  found  by  suitable  operations  on  a  range  image  of  the  object.  Many  possibilities 
remain  to  be  explored;  in  the  present  work  the  registered  range  and  intensity  images 
are  used  quite  naively. 

There  has  been  an  enormous  amount  of  work  done  in  the  area  of  recognition  of 
3-D  objects,  and  we  consider  some  of  the  relevant  topics  in  section  2.  In  section  3 
our  algorithm  for  matching  piecewise  linear  curves  in  three-space  is  described.    In 


section  4  the  range  sensor  which  provides  registered  range  and  intensity  images  of  a 
workspace  is  described.  In  section  5  a  series  of  experiments  designed  to  test  the 
algorithm  and  range  sensor  demonstrates  that  even  quite  similar  objects  consisting  a 
single  curve  in  three  space  can  be  distinguished.  The  results  are  summarized  in 
section  6. 


2.   Related  Work 

A  vast  literature  relevant  to  3-D  object  recognition  by  machine  exists,  and  has 
been  reviewed  recently  by  Besl  and  Jain  [BJ85].  In  this  section  we  focus  on  active 
range  sensors  which  have  so  far  provided  the  best  3-D  data  used  for  object 
identification.  A  number  of  additional  3-D  techniques  are  described  in  a  recent 
survey  by  Jarvis  [J83a].  We  give  particular  attention  to  ways  in  which  intrinsic 
curves  on  3-D  objects  have  been  used  or  observed,  and  to  exploitation  of  registered 
intensity  and  range  images. 

Shirai  and  Suwa  [SS71]  made  some  of  the  earliest  3-D  measurements  and  used 
the  data  to  recognize  polyhedrons.  The  plane-of-light  technique  yielded  single  stripe 
images  which  were  analyzed  for  end  points;  these  points  were  used  to  generate 
edges  which  were  analyzed  to  yield  planes,  and  objects  were  recognized  from  the 
shapes  and  spatial  relations  of  these  planes.  This  work  has  been  extended  to 
regions  composed  of  curved  surfaces  [OS83].  Similar  techniques  have  been  used  to 
generate  models  of  scenes  containing  polyhedrons  [H85]  or  polyhedrons  plus 
circular  cylinders  [PBAC75].  A  different  approach  to  recognizing  and  locating 
polyhedral  objects  has  been  pursued  by  Grimson  and  Lozano-Perez  [GLP84],  who 
examined  the  consistency  of  sparse  local  measurements  of  surface  positions  and 
normals  (such  as  might  be  provided  by  tactile  or  range  sensors)  with  constraints 
derived  from  mathematical  models  of  objects. 

Bolles  and  his  co-workers  have  recognized  an  industrial  casting  by  searching  an 
ordered  tree  of  features  extracted  from  plane-of-light  measurements 
[BF81][BHH83][HB84].  The  range  data  is  processed  at  a  low  level  to  find  edge 
points  which  are  partitioned  to  identify  those  that  lie  in  planes,  then  further 
characterized  as  straight  or  circular  edge  segments  and  by  the  nature  of  the  adjacent 
surfaces.  A  high  level  process  then  uses  features  already  identified  to  focus  its 
search  for  other  features  to  provide  information  about  the  current  interpretation  in  a 
'grow-a-match'  approach.  The  method  extracts  parametrized  curves  such  as  circular 
arcs  and  straight  lines,  and  may  not  be  able  to  handle  more  general  curves. 

Since  it  is  more  difficult  to  find  edges  in  objects  with  no  planar  surfaces, 
techniques  that  do  not  use  edges  have  been  developed.  Agin  and  Binford  [AB73] 
used  data  from  a  laser  plane-of-light  device  to  generate  cylinder  models  of  a  number 
of  objects  such  as  dolls.  Potmesil  [P83],  using  a  quadtree  representation  of  surface 
normals  as  a  data  structure,  generated  a  complete  surface  model  of  a  'blob-shaped 
balsa  model  of  a  car'  by  matching  and  merging  data  from  36  images  of  the  object 
illuminated  by  a  controlled  light  source.  Faugeras  and  his  co-workers  have  used  a 
technique  of  active  stereo  [Fea83,FH83]  to  collect  a  dense  set  of  points  on  the  entire 
surface  of  an  automobile  casting.  From  subsets  of  these  points  planes  were  grown 
and  used  to  match  two  views  of  overlapping  regions  with  moderate  success.  This 
work  has  been  extended  [F84]  to  fit  quadric  surfaces  to  range  measurements  of  a 
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specially  machined  object.  Faugeras  [F84]  notes  that  "line  primitives  such  as 
borders  between  regions,  lines  of  high  curvature,  etc.  are  also  most  probably 
useful"  for  object  recognition.  Ponce  and  Brady  [PB85],  using  dense  range  data 
from  several  sources,  have  tested  algorithms  designed  to  identify  and  isolate  a  set  of 
critical  surface  curves  which  include  regions  of  significant  surface  change. 

Laser  range  finders,  using  the  length  of  time  it  takes  for  light  to  travel  to  a 
point  on  an  object  and  back,  have  generated  range  images  of  complex  scenes 
[NBD77][LJ77]  at  a  range  of  several  meters  with  an  accuracy  of  several  centimeters. 
Jarvis  [J83b]  described  a  laser  sensor  which  he  expected  to  provide  an  image  of 
64x64  range  values  within  ±1/4  cm  in  40  seconds,  and  he  suggested  that  range 
information  might  help  to  segment  registered  images  to  be  made  using  a  color 
camera.  Recently  a  laser  sensor  has  been  developed  (see  discussion  of  the  ERIM 
sensor  in  [BJ85])  to  work  at  ranges  as  small  as  9  inches  with  very  good  resolution. 
These  sensors  have  apparently  not  been  used  for  object  recognition. 

Rocker  and  Kiessling  [RK75]  were  perhaps  the  first  to  exploit  registered 
intensity  and  range  images  when  they  used  edges  in  an  intensity  image  to  interpret 
images  of  multiple  stripes  projected  onto  polyhedrons.  In  [NBD77]  the  range  data 
is  used  to  correct  for  projective  distortions  and  make  designs  in  the  intensity  image 
recognizable;  later  in  [DNB79]  the  same  data  was  analyzed  using  the  intensity  and 
range  images  together  to  extract  planar  regions.  Gil  et  al.  [GMA83]  used  this  same 
data  to  extract  edges  which  could  be  found  in  both  range  and  intensity  images. 
Recently  Magee  and  Aggarwal  [MA84]  have  suggested  that  intensity  images  be  used 
to  guide  the  collection  of  range  data,  and  have  used  simulated  data  to  match  a 
relatively  small  number  of  feature  points  on  a  3-D  object  with  corresponding  points 
on  a  displaced  model. 

The  quantity  of  range  data  which  has  been  used  for  object  recognition  thus  far 
is  quite  small,  probably  in  part  because  until  recently  gathering  range  data  of  useful 
quality  was  tedious.  In  this  paper  we  show  data  extracted  from  more  than  two 
dozen  512x460  images.  Future  work  is  likely  to  include  processing  of  range  images 
to  discover  some  significant  features  (perhaps  along  the  lines  suggested  in  [PB85]) 
combined  with  processing  of  registered  intensity  images  (perhaps  along  lines 
suggested  by  Haralick  et  al.  [HWL83])  to  bring  out  other  significant  features,  with 
absolute  shape  and  geometric  inter-relationships  easily  available  for  all  features.  It 
would  appear  that  range  and  intensity  used  together  are  more  powerful  than  both 
images  used  separately,  and  that  as  range  information  becomes  more  easily 
available  it  can  be  used  effectively. 


3.   Fast  Curve  Matching  In  3  Dimensions  By  Least  Squares  Fitting 

This  section  reviews  the  fast  technique  for  matching  (a  portion  oO  an  observed 
curve  C  to  be  identified  against  a  corresponding  portion  of  a  (previously  obtained) 
model  curve  C,  as  previously  presented  in  [SchSh85].  The  curves  C  and  C  are 
represented  in  identical  fashion.  Model  curves  are  typically  closed  curves,  although 
the  algorithm  may  be  adapted  for  model  curves  which  are  open.  The  algorithm 
calculates  the  position  and  orientation  of  C  and  also  the  least  squares  difference 
between  C  and  C.  If  C  is  a  curve  fragment  which  is  the  visible  portion  of  an 
occluded  curve,  the  algorithm  matches  C  to  the  section  of  C  that  is  closest  to  C.  The 


least  squares  difference  indicates  the  quality  of  the  match,  and  may  be  used  to 
identify  the  curve  from  a  collection  of  model  curves. 

The  curves  C  and  C  are  parametrized  by  arclength,  and  so  the  algorithm  is  not 
scale  invariant.  However,  because  the  depth  sensor  supplies  real  x,y,z  coordinate 
positions,  size  ambiguity  is  not  a  concern.  The  success  of  the  matching  algorithm 
depends  primarily  on  the  accuracy  of  the  curve  arclengths.  The  smoothing  technique 
used,  which  is  described  in  Section  3.2  of  [SchSh85]),  is  designed  to  preserve  the 
arclength  of  curves  within  a  uniform  maximal  error  which  depends  linearly  on  a 
noise  parameter  €. 

Matching  calls  for  determination  of  the  offset  Sq  and  the  Euclidean 
transformation  E  for  which  the  curves  C'{s  +  Sq)  and  EC{s)  are  closest  to  one  another 
in  the  L?  norm,  i.e.  in  the  least  squares  sense.  More  specifically,  let  the  curves  C 
and  C  be  represented  by  a  sequences  of  evenly  spaced  points  (u^)"=i  and  (Vj)"=i 
respectively.  Assume  first  that  the  whole  of  C  is  visible,  the  sampling  starts  at 
corresponding  points  along  C  and  C,  and  that  m  =  n  (which  is  to  say,  the  two 
curves  C  and  C  are  of  equal  length;  of  course,  if  the  lengths  differ  significantly, 
there  is  no  point  in  matching  them).  The  matching  thus  amounts  to  finding  a 
Euclidean  motion  E  of  3-space  which  will  minimize  the  /'  distance  between  the 
sequences  {EmjYj^^  and  {yj)"={,  i.e.  one  needs  to  compute 

A  =  min   i:|£u^.  -  v/. 

To  simplify  the  calculation,  C  is  first  translated  so  that  its  centroid  lies  at  the  origin, 
giving 

i  u,  =  0. 
Next  E  is  written  as  £u  =  /fu  +  a,  where  i?  is  a  pure  rotation.   Then 

n 

A  =  min     5^1i?u,  +  a  -  v,p  = 


mm 

A.  a 


S  KF    +  /.lap  -  2X  a-v,.  +  2  |u;F  +  22  a-/?u^.  -  2^  Ruy^j 


But 

2  a/?u^  =  bR(2xxj)  =  0  . 

Hence  a  and  R  appear  independently  in  A  and  their  contributions  can  be   minimized 
separately.   To  minimize  over  a  simply  put 

1     " 

«  =    7  2   ^;- 

As  to  ^,  it  is  necessary  to  compute 


8  =  max   X  R^r^i- 
To  find  8  one  first  calculates  the  matrix  A  given  by 
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for  /',;'  =  1,2,3,  where  z^'^  denotes  the  /-th  coordinate  of  the  vector  z.  In  terms  of  the 
matrix  A  one  can  express  8  as 

8  =  max     tr(RA). 

R 

To  maximize  tr(RA)  first  assume  A  to  be  non-singular;  then  A    can  be  decomposed  as 

-i  i 

A  =  QH ,  where  Q  =  A{A*A)  ^  is  a  pure  rotation,  and  H  =  (A*A)^  is  positive  definite 
and  symmetric.  This  gives 

i 
8  =  max    tr(RQH)  =  max    tr{RH)  =  tr{H)  =  tr({A*A)^). 

K  R 

Indeed,  since  the  trace  is  invariant  under  rotation,  one  can  assume  without  loss  of 
generality  that  H  is  diagonal.  But  for  a  diagonal  positive  definite  matrix  (X,)  and  a 
rotation  matrix  (r,_,),  the  trace  of  their  product  2  ^/''i.  can  be  no  larger  than  2  ^i  ^nd 
can  assume  this  value  only  when  r,^  =  8|_,.  An  identical  result  for  a  singular  A  follows 
easily  by  continuity  arguments.  Altogether  one  has 

A  =  i  K/-  -  -li  yj?  +  i  \uj\'  -  2  tr  ((A'A)^)  (3.1) 

showing  that  the  optimal  rotated  match  between  C  and  C  can  be  found  in  time 
linear  in  n.  Note  that  the  trace  calculation  involves  3x3  matrices,  and  so  can  be 
accomplished  in  constant  time,  after  the  matrix  A  has  been  calculated  in  0{^n)  time; 
more  on  the  technique  being  used  to  calculate  this  trace  can  be  found  in  [SchSh85]. 

If  C  is  partially  occluded  (thus  no  longer  assuming  m  =  «),  or  even  if  the  whole 
of  C  is  visible,  but  the  starting  points  for  the  samplings  along  C  and  C  do  not 
necessarily  match  one  another,  then  one  must  match  the  sequence  (";)"=  i  to  each  of 
the  contiguous  subsequences  (vy+^)"=,  of  the  (circular)  sequence  (▼;)". j,  for 
rf  =  0,  .  .  .  ,m-l.  (We  assume  that  m  ^  n,  for  otherwise  even  the  visible  portion  of 
C  is  too  long  to  match  C.) 

For  each  such  d,  eq.  3.1  thus  becomes 

A.  =     'i"    hP  -  f  I   2     v,P  +  i  KF  -  2  tr  ((a;a,)^)  .  (3.2) 


=d^\  "    j=d*\  >=1 


where 


The  minimum  of  the  values  A^,  <f  =  0,  .  .  .  ,m- 1,  can  be  found  in  time  0{m  log  m), 
as  follows.  The  first  three  terms  of  eq.  3.2  can  be  updated  in  constant  time  when  d 
is  increased  by  1,  so  that  the  total  cost  of  their  calculation  for  all  d  is  only  0{m).  As 
to  the  fourth  term,  it  requires  a  series  of  constant  time  calculations  on  a  sequence  of 
matrices  A^,  each  of  whose  entries  forms  a  corresponding  sequence  of  convolutions 
of  two  vectors  u^'^  and  y'-^\  It  follows  that  the  entire  sequence  of  matrices  A^  can  be 
calculated    in    time    0(m  log  m),    using    a    fast    Fourier    transform    algorithm    for 
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computing  the  required  convolutions,  after  which  the  values  of  the  corresponding 
traces  can  be  all  calculated  in  0(m)  time. 

Thus  one  can  find  the  offset  d  which  minimizes  A^,  and  then  face  the  task  of 
calculating  the  corresponding  Euclidean  transformation  £u  =  /?u  +  a  which  yields 
the  minimum  least-squares  distance  between  the  two  sequences  u  and  v.  As  shown 
above,  the  displacement  a  is  simply 


1     " 

=  -  2  yj.. 


n  ,.=  , 

assuming,  as  before,  that  the  centroid  of  u  lies  at  the  origin.  The  rotation  R  which 
maximizes  tr  A  (where  we  put  A  =  Aj)  is  found  by  noting  that  R  =  Q*  where  Q  is  the 
rotation  appearing  in  the  decomposition  A  =  QH  given  above,  and  then  calculating  Q 
using  standard  techniques  of  numerical  analysis  [SchSh85]. 

4.   The  Range  Sensor 

The  three  dimensional  information  used  in  these  experiments  was  provided  by 
a  range  sensor  [S83][BS84][B84][BS86].  The  device  implements  a  method  of  optical 
triangulation  using  a  structured  beam  of  light  and  a  video  camera,  together  with  an 
image  processing  system  able  to  digitize  images  and  perform  simple  operations  such 
as  adding  or  multiplying  two  images,  or  modifying  an  image  using  a  look-up  table. 
The  output  of  the  device  is  a  range  image  in  which  the  value  at  each  pixel  represents 
the  distance  from  a  plane  perpendicular  to  the  camera  axis  of  whatever  is  imaged 
onto  that  pixel.  In  addition  to  the  range  image  a  registered  intensity  image  is 
produced. 

The  range  sensor  is  similar  to  well  known  plane-of-light  range  sensors 
[SS71][AB73][HB84]  in  that  the  location  of  an  object  is  determined  by  the 
intersection  of  a  (vertical)  plane  which  identifies  a  direction  from  a  projector  and  a 
ray  which  identifies  a  direction  from  a  camera.  The  device  differs  from  plane-of- 
light  triangulation  devices  in  that  no  scanning  of  light  planes  is  needed  to  generate  a 
complete  range  image.  The  working  area  of  the  sensor  is  illuminated  successively 
by  two  beams  of  light  which  encode  a  continuum  of  planes  simultaneously,  each 
plane  corresponding  to  a  gray  level  in  the  ratio  image  formed  as  an  intermediate 
step.  The  method  is  analogous  to  the  technique  which  uses  multiple  projected 
planes  (e.g.  [ISM84]),  and  in  fact  the  calibration  and  calculation  procedure  used 
here  can  be  applied  there,  but  the  continuous  ratio  avoids  discretization  problems. 
With  this  technique  a  512  x  460  x  (nominally)  12  bit  range  image  is  generated  in 
forty  seconds. 

4.1.   Theory 

The  principle  of  this  novel  technique  can  be  explained  briefly  with  reference  to 
a  two-dimensional  version  of  the  sensor  sketched  in  Fig  4.1.  An  object  is  illuminated 
by  a  projector  (e.g.  a  bright  point  source  of  light)  and  is  viewed  by  a  camera  which 
provides  a  digitized  image  of  the  scene.  It  is  assumed  for  the  moment  that  each 
digitized  value  is  proportional  to  the  average  intensity  of  the  picture  element  (pixel) 
it  represents,  and  that  each  pixel  corresponds  to  a  known  direction  from  the  camera. 
If  the  intensity  of  the  light  is  varied  across  the  beam  (e.g.  by  placing  a  suitable  filter 
across  the  beam  in  front  of  the  light  source)  it  should  be  possible  to  identify  the 


location  of  an  object  in  the  beam  from  the  intensity  of  the  light  reflected  to  the 
camera,  since  an  object  in  the  bright  part  of  the  beam  would  appear  brighter  than  it 
would  sitting  in  the  darker  part  of  the  beam  (assuming  the  same  distance  from  the 
projector  and  the  same  orientation  to  the  projector  and  camera). 

Some  refinement  of  this  simple  method  is  needed,  since  the  intensity  of  light 
recorded  at  the  camera  depends  on  the  reflectivity  of  the  object  and  the  position  and 
orientation  of  the  surface  with  respect  to  the  light  source  and  camera.  However, 
variations  in  the  intensity  due  to  these  factors  can  be  cancelled  by  making  a  second 
digitized  image  with  the  scene  illuminated  by  a  beam  of  different  intensity 
distribution  and  dividing  these  two  digitized  images  pixel-by-pixel.  Since  camera  and 
object  have  not  moved,  exactly  the  same  part  of  the  scene  is  imaged  onto  each  pixel 
in  both  exposures. 

The  intensity  /,_,  of  light  reflected  by  a  small  region  of  the  surface  to  the  pixel 
I,;'  can  be  expressed  as 

A;    =    ^0  /  ^reflta  ^ angle  (4.1) 

where  /q  represents  the  intensity  of  light  which  would  be  projected  onto  the  region 
with  no  filter;  /  represents  a  reduction  caused  by  the  filter,  and  F^^^^,  and  F^„^;,  are 
values  between  zero  and  one  which  represent  the  reduction  of  the  reflected  light 
intensity  due  to  surface  reflectivity  and  orientation.  The  exact  form  of  these  f"s  is 
unimportant  since  they  are  identical  for  both  intensity  images.  Only  the  filter 
function  /  changes  to  make  one  intensity  image  different  from  the  other.  The  ratio 
at  any  pixel  is  thus 

'^        /■* 
^.>  =  7  =  >•  (4-2) 

where  f^  and  /^  are  the  reductions  in  the  light  intensity  projected  onto  the  object  in 
the  region  imaged  in  camera  pixel  ij.  When  the  filter  functions  are  appropriately 
chosen  this  ratio  can  be  made  to  vary  across  the  projector  beam  in  a  strictly 
monotonic  manner.  Since  eq.  4.2  is  good  for  all  pixels,  R,j  represents  a  gray  scale 
image  of  ratio  values,  and  every  ratio  value  is  related  uniquely  to  a  direction  from 
the  projector.  This  ratio  image  can  be  thought  of  as  being  the  image  formed  if  the 
object  were  photographed  using  a  hypothetical  camera  and  projector  capable  of 
using  'ratio  rays'  whose  'intensity'  is  given  by  the  ratio  of  the  filters  in  each 
direction  from  the  projector.  The  usual  intensity  information  does  not  exist  in  the 
ratio  image. 

Note  that  the  intensities  /  above  refer  only  to  the  light  projected  through  the 
filters.  Under  normal  working  conditions  intensity  images  include  a  contribution 
from  room  lights.  Therefore  an  image  of  the  work  area  in  ambient  light  (projector 
beam  blocked)  is  made,  and  this  image  is  subtracted  from  the  images  made  with  the 
two  projected  beams  to  make  the  intensity  images  used  in  the  calculations.  This 
procedure  also  removes  camera  zero  offsets  which  may  differ  from  pixel  to  pixel. 

For  the  range  sensor  to  work  properly  the  filters  must  be  chosen  so  that  there 
is  a  one-to-one  correspondence  between  observed  ratio  values  and  directions  from 
the  projector.  For  example,  the  ratio  would  vary  linearly  across  the  beam  if  one 
filter  were  clear  on  one  end  and  varied  linearly  to  20'7c  transmitting  on  the  other 
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and  the  second  filter  were  completely  clear.  When  the  ratio  varies  monotonically 
across  the  beam  it  also  varies  monotonically  along  the  optical  axis  of  the  camera. 
With  suitable  observations  a  look-up  table  can  be  created  which  gives  distance  along 
the  camera  axis  for  any  input  ratio  value;  generating  this  look-up  table  is  an 
important  part  of  the  calibration  of  the  sensor. 

With  the  look-up  table  suggested  above  the  range  for  objects  along  the  camera 
axis  is  directly  computed.  Moreover,  the  range  for  objects  anywhere  in  the 
workspace  can  be  determined  using  the  same  look-up  table  and  a  suitable 
correction.  The  quantities  used  in  the  derivation  of  this  correction  are  defined  in 
Fig.  4.2.  The  camera  lies  at  the  origin  of  the  coordinate  system  looking  along  the 
z-axis,  and  rays  from  the  projector  diverge  from  the  point  jcq.zq.  An  arbitrary  point 
in  the  workspace  x,z  is  determined  by  the  intersection  of  a  ray  from  the  camera  at 
angle  6  from  the  z-axis  and  a  ray  from  the  projector  which  intersects  the  z-axis  at 
0,d.  Recall  that  any  ray  from  the  projector  consists  of  all  points  having  a  single 
ratio  value,  thus  d  is  known  from  the  ratio  observed  on  the  object  at  x,z  and  the 
look-up  table. 

When  line  segments  from  x,z  and  Xq,zq  are  drawn  perpendicular  to  the  z-axis  it 
is  clear  by  similar  triangles  that 

^  =  ^-  (4.3) 

X  Xq 

These  quantities  are  coordinate  values,  and  can  be  negative,  as  is  Xq  here.  This 
expression  can  be  re-written  as 

z  —  X  =  a 


or 


^0"*^  X. 


z[l  -    -^ ^]  =   d 

Xq       z 

or 


z  = 


^^tan(e)  (4.4) 


where  x/z  has  been  written  as  tan(e).  Thus  the  actual  range  z  for  any  point  in  the 
workspace  can  be  calculated  from  the  observed  ratio  (which  gave  d  from  the  look- 
up table),  the  known  projector  location  xq.zq,  and  the  direction  from  the  camera  6 
which  is  known  from  the  pixel  on  which  the  point  x,z  is  imaged. 

Because  the  computation  is  carried  out  on  a  VICOM  image  processor  which 
has  only  12  bits  available  for  computation,  eq.  4.4  is  rewritten  so  the  available  bits 
span  the  workspace  by  translating  the  origin  to  the  point  in  the  workspace  closest  to 
the  camera  and  scaling  the  range  units  so  the  furthest  point  in  the  workspace  is 
represented  by  the  VICOM's  largest  positive  integer.  The  translation  of  the  origin 
changes  the  form  of  eq.  4.4  slightly.  The  new  ranges  Z  are  related  to  the  old  by 
z  =  Z  +  tfo-   Making  this  substitution  in  eq.  4.4  results  in 
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Z  =  i<t)^,  (4.5) 

where  p  is  the  observed  ratio  value,  8(p)  is  the  appropriate  range  look-up  table  and 

Zo-8(p) 
C  =  tan(e)  is  a  small  correction.    Zq  is  the  projector  location  in  the  new 

range  coordinates.  In  the  present  implementation  the  2048  positive  range  values 
span  a  workspace  of  somewhat  less  than  one  meter,  resulting  in  a  quantization  limit 
to  accuracy  of  about  0.5mm. 

Discussion  of  the  range  sensor  has  thus  far  been  explicitly  that  of  the  2-D 
version  shown  in  Figs.  4.1  and  4.2.  Extension  to  3-D  is  trivial  and  the  same 
equations  are  valid  provided  that  the  optical  axes  of  the  camera  and  projector  are 
coplanar  and  the  filters  are  designed  to  project  planes  of  constant  ratio 
perpendicular  to  this  plane.  The  location  in  space  of  an  imaged  region  is 
determined  by  the  intersection  of  a  ray  from  the  camera  with  a  vertical  plane  from 
the  projector.  The  plane  is  identified  by  the  ratio  value  p  and  Eq.  4.5  is  used  to 
determine  the  range.  X  and  Y  coordinates  are  determined  from  the  range  and  the 
observed  angles  from  the  camera  axis. 

4.2.    Implementation 

The  equipment  used  for  the  experiments  described  in  this  paper  consisted  of  a 
Kodak  Ektagraphic  3B  slide  projector  with  a  100  to  150mm  f/3.5  zoom  lens,  a 
General  Electric  4TN2505  solid  state  video  camera  with  a  Tokina  f/1.4  16mm  lens, 
and  a  VICOM  image  processor  (with  firmware  operating  system).  A  VAX  750 
running  Unix  4.2  served  as  host  to  the  VICOM.  A  special  purpose  interpreter  VSH 
[CH85]  greatly  facilitated  host/VICOM  communication.  The  host  computer  sent 
command  bytes  to  an  ADC-1  box  [RMS]  to  control  the  slide  projector.  The  graded, 
neutral  density  filters  [REI]  were  manufactured  using  a  process  which  ensured  that 
the  iso-density  contours  were  parallel;  the  filters  used  in  these  experiments  varied 
linearly  in  optical  density  from  about  20%  transmission  at  the  dark  end  to  clear 
glass  at  the  bright  end.  The  glass  filters  were  mounted  in  modified  2  inch  plastic 
slide  mounts. 

The  work  area  of  the  range  sensor  is  the  area  which  is  both  illuminated  by  the 
projector  and  viewed  by  the  camera.  The  camera  and  projector  were  positioned  so 
that  their  optical  axes  were  coplanar  and  so  that  both  the  iso-ratio  planes  and  the 
vertical  camera  direction  were  perpendicular  to  this  plane.  The  angle  between  the 
optical  axes  was  about  32°,  a  compromise  between  the  conflicting  goals  of  good 
range  resolution  and  shadow-free  images.  The  camera  was  about  80  cm  from  the 
center  of  the  work  area  and  the  projector  about  140  cm  (both  distances  measured  to 
the  front  of  the  lenses).  Range  values  were  returned  for  surface  points  in  an  86  cm 
deep  region  starting  41  cm  from  the  camera;  however,  the  most  reliable 
measurements  were  made  in  an  area  approximately  30  cm  in  range  by  20  cm  wide 
near  the  center  of  the  workspace.  The  range  sensor  was  horizontal  and  the 
workspace  was  a  table  top,  but  the  device  can  be  configured  for  many  different 
applications.  The  working  coordinates  were  x  (horizontal),  y  (vertical),  and  z 
(range,  parallel  to  the  camera  axis). 

Calibration  of  the  device  involved  determining  the  6(p)  function  and  the 
location  of  the  projector  with  respect  to  the  camera.    (Recall  that  8(p)  is  the  range 
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along  the  camera  axis  which  corresponds  to  an  observed  ratio  p;  see  eq.  4.5.)  The 
location  of  the  projector  was  determined  by  projecting  a  single  vertical  plane  onto  a 
screen  at  three  different  distances  from  the  camera,  and  processing  images  of  these 
projections  on  the  VAX  together  with  measurements  of  the  screen  locations  relative 
to  the  camera  and  projector. 

The  function  8(p)  was  determined  as  follows.  At  each  of  the  above  three 
screen  locations  a  ratio  image  was  made,  thus  generating  a  function  of  ratio  vs  x  for 
each  distance.  After  the  location  of  the  projector  was  computed  these  three  ratio  vs 
X  functions  were  combined  into  a  single  ratio  vs  distance  function  which  was 
inverted  to  give  a  function  of  distance  vs  ratio  along  the  centerline  of  the  camera. 
This,  with  the  distance  values  appropriately  converted,  became  the  desired  B(p) 
look-up  table. 

The  program  which  guided  the  calibration  process  and  computed  the  8(p) 
function  also  placed  constants  needed  for  the  VICOM  calculation  in  the  look-up 
table  at  ratio  values  outside  the  normal  range,  and  made  other  calibration 
information  (such  as  the  vertical  and  horizontal  fields  of  view  of  the  camera)  easily 
available  for  routines  which  later  converted  range  images  to  x,y,z  coordinates  in 
physical  units.  The  set-up  and  basic  calibration  of  the  range  sensor  took  about  30 
minutes. 

After  this  calibration  procedure  fine-tuning  parameters  were  determined  to 
correct  for  small  systematic  deviations  in  the  observed  locations  of  objects  placed  at 
known  grid  points  in  the  workspace.  These  corrections  were  installed  in  the 
conversion  routine  to  improve  the  grasping  accuracy  of  a  robot  interfaced  with  the 
range  sensor,  and  are  probably  not  relevant  to  the  matching  experiments  since  the 
fine-tuning  does  not  change  shapes  over  small  regions. 

Since  a  new  calibration  would  be  needed  if  the  camera  moved  relative  to  the 
projector  both  components  were  fixed  in  place.  The  projector  zoom  lens  was  kept 
at  its  most  wide  open  setting,  and  the  focus  settings  were  always  set  as  they  were 
during  the  calibration  (camera  and  projector  focused  on  the  center  of  the 
workspace).  The  camera  diaphragm  was  set  for  proper  exposure  before  each  range 
image.   Range  measurements  taken  months  apart  produced  comparable  data. 

A  range  image  was  made  by  digitizing  the  scene  (objects  and  background)  in 
ambient  light  (overhead  fluorescent  lamps),  in  the  illumination  of  the  beam 
modified  by  the  first  filter,  and  in  the  beam  modified  by  the  second  filter.  In  each 
of  these  conditions  eight  digitizations  were  made  and  averaged  to  reduce  the  effect 
of  random  camera  noise.  All  computations  (combining  the  intensity  images  to  make 
the  ratio  image,  converting  the  ratio  image  to  a  range  image)  were  performed  on 
the  VICOM.  The  final  intensity  image  was  made  by  adding  the  two  intensity  images 
after  the  ambient  light  had  been  subtracted  from  them.  It  typically  required  21 
seconds  to  make  the  digitizations  (including  the  second  or  so  it  took  to  change  filters 
under  computer  control),  and  a  further  18  seconds  for  the  computation  of  the  range 
image  using  eq.  4.5.  The  process  was  controlled  by  a  routine  running  on  the  VAX 
which  sent  commands  by  serial  lines  to  the  VICOM  and  to  the  hardware  which 
controlled  the  slide  projector. 
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4.3.    Performance 

A  number  of  factors  affect  the  quality  of  the  range  image.  For  example,  noise 
generated  by  the  camera  causes  random  variations  in  the  recorded  intensity;  in 
regions  of  low  intensity  the  quantization  steps  are  relatively  larger  than  in  regions  of 
high  intensity;  the  camera  appears  to  have  a  slightly  non-linear  response  and 
unevenness  in  the  sensitivity  of  adjacent  pixels.  The  following  experiment  was 
made  to  investigate  the  uncertainty  in  the  final  range  images. 

A  white  screen  was  placed  perpendicular  to  the  camera  axis  and  two  range 
images  of  it  were  made.  The  magnitude  of  the  difference  image  of  these  was 
computed,  averaged  over  16x16  pixel  blocks,  and  recorded  as  a  32x30  matrix  so  the 
distribution  of  differences  between  the  images  could  be  examined  for  spatial 
variation.  The  values  in  this  noise  matrix  varied  from  less  than  0.5  mm  to  over  2 
mm,  with  one  column  (in  a  region  of  the  camera  retina  which  is  particularly  noisy) 
having  values  near  3mm.  Typical  values  were  about  1.5  mm  on  the  left  of  the 
image  and  2.1  mm  on  the  right.  Thus  the  range  recorded  at  a  pixel  was  observed  to 
change  by  an  average  of  something  less  than  2mm  from  one  range  image  to  the 
next. 

Both  512x460  pixel  range  images  were  also  reduced  to  32x30  images  by 
averaging  16x16  pixel  blocks.  In  these  reduced  images  corresponding  blocks 
differed  by  typically  0.4mm  in  range,  even  in  the  noisy  regions.  This,  and  the 
correlation  between  adjacent  blocks,  suggests  that  only  part  of  the  2mm  average 
deviation  found  in  the  previous  paragraph  can  be  ascribed  to  random  camera  noise. 
Examples  of  noise  sources  which  affect  large  regions  of  the  image  are  variations  in 
the  projector  intensity  at  120  hz  and  at  much  lower  (several  second)  rates.  In 
addition  to  temporal  variation  the  reduced  range  images  showed  spatial  variation, 
typically  0.8mm  from  one  16x16  block  to  the  next,  but  sometimes  as  much  as  2mm. 
Since  the  screen  was  flat  and  perpendicular  to  the  camera  axis  the  reduced  range 
images  should  have  shown  uniform  range.  The  origin  of  these  variations  is  not 
entirely  clear,  although  dirt  and  pinholes  on  the  filters  have  been  observed  to  cause 
large  local  variations  in  the  recorded  range. 

The  range  sensor  in  this  implementation  has  proven  insensitive  to  uncertainties 
in  the  calibration  process.  Small  errors  in  the  focus  adjustment  of  the  camera  or 
projector  are  not  noticeable,  but  gross  errors  distort  the  test  figures  (see  section  5) 
sufficiently  to  cause  mismatches  in  the  balloon  figures.  The  slide  projector  positions 
the  filters  reproducibly  within  less  than  1/2  pixel  in  the  work  area  as  imaged  by  the 
camera. 

Test  calibrations  made  with  deliberate  errors  of  several  centimeters  in  the 
reported  position  of  the  camera  or  projector  did  not  cause  noticeable  distortion. 
This  is  important  as  a  demonstration  that  it  is  not  necessary  to  determine  the 
location  of  the  projector  (which  was  assumed  to  act  as  a  point  in  the  theoretical 
section)  with  extreme  accuracy.  In  fact  there  is  no  single  point  through  which  all 
projected  rays  pass;  it  appears  that  an  effective  divergence  point  exists  slightly  in 
front  of  the  lens. 

Depth  of  field  has  not  been  a  problem.  The  camera  lens  is  set  typically  at  f/4, 
resulting  in  a  depth  of  field  comparable  to  the  work  area  of  the  sensor.  The  depth 
of  field  of  the  projector  (f/3.5  lens)  is  similarly  comparable  to  the  work  area.    We 
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note  that  if  the  fihers  change  linearly  in  transmissivity  a  change  in  the  intensity 
distribution  would  not  be  caused  by  an  out  of  focus  condition.  In  practice  ratio  rays 
have  been  found  to  be  straight  lines. 

Limitations  of  the  camera  directly  affect  the  performance  of  the  range  sensor. 
The  ideal  camera  would  have  minimum  noise,  good  linearity,  perfectly  local 
response  and  no  geometric  distortion.  Real  cameras  fall  short  of  these  ideals  in 
every  category,  and  the  optimum  balance  of  defects  is  not  clear.  The  accuracy  and 
resolution  of  the  present  sensor  are  limited  primarily  by  deficiencies  of  the  camera. 

Care  was  taken  in  the  experiments  of  section  5  to  avoid  instances  of  'secondary 
illumination',  which  occurs  when  light  is  reflected  from  one  region  of  the  workspace 
to  another  before  being  reflected  to  the  camera.  Secondary  illumination  causes 
problems  in  some  form  with  any  active  range  sensor,  and  the  range  sensor 
described  here  may  be  more  sensitive  to  this  problem  than  other  sensors.  It  may  be 
possible  to  detect  the  presence  of  secondary  illumination  at  the  sensor  level,  but  a 
simple  solution  may  not  be  easy  to  install. 

5.   Experimental  Identification  of  Curves  in  3-Space 

A  series  of  experiments  was  carried  out  to  test  the  matching  algorithm  for 
generally  correct  behavior  with  real  data  and  to  test  the  quality  of  the  geometric 
data  provided  by  the  range  sensor.  Piecewise  linear  approximations  to  curves  in 
three  space  were  extracted  from  registered  range  and  intensity  images,  and  were 
matched  against  other  curves  made  from  the  same  object  in  different  locations  and 
against  curves  made  from  different  objects. 

5.1.   The  Curves 

The  curves  used  in  this  experiment  were  the  edges  of  construction  paper 
figures  glued  to  the  surface  of  terra-cotta  flower  pots.  Yellow  and  blue  paper 
figures  were  used  to  enhance  contrast  by  varying  the  reflectivity  of  the  pots,  the 
yellow  appearing  brighter  and  the  blue  darker  than  the  pot.  The  edges  of  the  paper 
figures  were  easily  found  in  the  intensity  image,  and  the  corresponding  pixels  in  the 
range  image  were  used  to  obtain  a  representation  of  the  curve  in  space.  The  figures 
were  placed  on  the  surfaces  of  the  pots  to  generate  durable  non-planar  curves  while 
avoiding  range  discontinuities  at  the  edges  of  the  figures. 

An  alternative  method  would  have  been  to  let  figures  stand  freely  in  the  work 
area  and  to  use  discontinuities  in  the  range  image  to  determine  the  curves. 
However,  because  pixels  have  finite  extent  a  pixel  on  the  edge  of  a  free-standing 
figure  would  receive  light  from  the  figure  and  also  from  the  background.  As  a 
result  values  in  a  range  image  at  the  edge  of  an  object  are  often  a  value  intermediate 
between  the  range  of  the  object  and  the  range  of  the  background,  since  the  value  for 
any  pixel  is  determined  by  the  amount  of  each  range  imaged  onto  the  pixel.  Such 
images  must  be  handled  in  special  ways  to  obtain  an  accurate  representation  of  these 
three-space  'jump-boundaries'.  Since  the  purpose  of  these  experiments  was  to  test 
the  sensor  and  matching  algorithm,  curves  were  generated  in  a  manner  which 
avoided  these  known  difficulties. 

Three  sets  of  figures  were  used  in  the  experiments  reported  here.  The  first  set 
contained  six  general  shapes,  the  second  three  balloons  differing  only  in  size,  and 
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the  third  three  rectangles  of  identical  size  which  were  attached  to  regions  of 
different  curvature  of  the  flower  pots.  The  surfaces  of  the  pots  (one  large  and  two 
small)  approach  the  shape  of  a  right  circular  cone  which  has  the  angle  12.4  degrees 
between  the  axis  and  the  surface  generating  ray.  the  second  and  third  sets  were 
designed  to  investigate  the  limits  of  the  range  sensor  and  matching  algorithm.  Table 
5.1  summarizes  the  figures. 


Name 

color 

pot 

radius(cm) 

bean 

yellow 

small 

4.4 

palette 

blue 

small 

4.4 

pansy 

blue 

small 

4.5 

peanut 

blue 

large 

6.0 

sail 

yellow 

large 

6.7 

sundrop 

yellow 

small 

4.5 

bala 

blue 

small 

4.5 

balb 

yellow 

large 

6.5 

bale 

blue 

large 

6.4 

tecs 

blue 

small 

4.2 

reel 

blue 

large 

6.9 

recv 

blue 

small 

4.5 

Table  5.1     Properties  of  Figures 

The  general  figures  were  cut  from  construction  paper  and  were  intended  to  be 
of  comparable  size  and  intuitively  different  in  shape.  These  figures,  as  projected 
onto  the  camera  retina  and  after  smoothing,  are  shown  in  Fig.  5.1,  where  each  box 
represents  a  small  square  region  of  the  image  centered  on  the  figure.  The  figures 
were  generated  not  so  much  to  demonstrate  completeness  as  to  disclose  unsuspected 
difficulties  in  the  matching  protocol. 

The  balloon  figures  were  identified  as  bala,  balb,  and  bale  in  order  of 
increasing  size.  The  shape,  two  circular  arcs  joined  by  their  common  tangents,  is 
shown  in  Fig.  5.2.  A  distinguishing  feature  of  the  figures  was  the  radius  of  the 
larger  circle:  for  bala  this  was  2.4cm,  for  balb  2.60cm,  and  for  bale  2.85cm.  The 
figures  differ  by  about  9%  in  linear  dimension.  Balb  was  made  from  yellow  paper 
to  make  it  easier  for  the  experimenter  to  distinguish  between  the  two  balloons  on 
the  large  pot.  Projections  of  these  figures  onto  the  camera  retina,  shown  in  Fig. 
5.3,  varied  considerably  in  size  and  shape  because  the  figures  were  placed  in 
different  locations  and  orientations  in  the  workspace. 

The  rectangle  figures,  identified  as  recs,  reel,  and  recv  as  mnemonics  for  small, 
large,  and  vertical,  were  3.25  x  5.95  cm  within  an  estimated  uncertainty  of  0.03  cm. 
Recs  was  placed  on  a  small  pot  with  its  long  side  curving  around  the  pot,  reel  was 
placed  on  the  large  pot  with  its  long  side  curving  around  the  pot,  and  recv  was 
placed  on  a  small  pot  with  its  short  side  curving  around  the  pot.  Fig.  5.4  shows 
smoothed  and  unsmoothed  versions  of  projections  of  these  figures. 
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5.2.    Data  Gathering  and  Extraction 

The  procedure  for  making  measurements  was  the  same  at  all  times,  that  is, 
there  was  no  distinction  between  measurements  for  'model'  curves  or  'observed' 
curves.  A  pot  was  placed  in  the  work  area  of  the  range  sensor  with  part  or  all  of  a 
selected  figure  illuminated  and  visible  to  the  camera.  The  camera  diaphragm  was 
adjusted  to  let  in  as  much  light  as  possible  without  exceeding  the  maximum  output 
of  the  camera.  The  registered  range  and  intensity  images  were  made  on  the 
VICOM  as  described  in  section  4.2,  and  these  were  then  processed  on  the  VICOM 
to  identify  those  points  which  responded  to  an  edge  detector  in  the  intensity  image 
and  were  not  near  edges  found  in  the  range  image.  An  image  having  the  range 
values  in  these  pixels  and  zeros  in  all  other  pixels  was  then  sent  to  the  VAX  where 
a  line-following  algorithm  made  lists  of  connected  points.  The  output  lists  of  points 
were  smoothed,  converted  to  physical  coordinate  values,  and  used  in  the  matching 
program. 

Edges  which  separate  an  object  from  the  background  are  easily  found  in  a 
range  image  using  almost  any  edge  detector.  In  this  implementation  the  Sobel 
Magnitude  operator  implemented  as  a  VICOM  command  was  used,  and  the  pixels 
identified  by  this  operator  were  made  into  a  mask  representing  the  range  edges. 
Edges  in  the  intensity  image  marking  the  boundaries  of  the  paper  figures  were 
somewhat  more  difficult  to  extract,  and  an  implementation  of  a  difference  of 
Gaussians  algorithm  [HL]  was  used  to  make  a  mask  of  edges  in  the  intensity  image. 
This  implementation  provides  edges  which  are  one  pixel  wide.  From  these  two 
masks  a  mask  of  all  intensity  edges  which  were  not  range  edges  was  made  to 
identify  the  appropriate  pixels  in  the  range  image.  The  range  values  in  the  final 
image  were  intended  to  be  only  those  along  the  edges  of  the  paper  figures,  but  some 
range  values  from  noise  pixels  generated  by  the  difference  of  Gaussians  edge 
operator  and  from  regions  where  the  curvature  of  the  object  changed  rapidly  were 
also  included.  For  example,  the  lip  of  a  flower  pot  and  the  edge  where  the  side  and 
bottom  meet  were  often  represented.  While  these  could  be  significant  curves  for  the 
recognition  of  flower  pots,  in  this  report  only  the  curves  from  the  boundaries  of  the 
paper  figures  are  considered. 

The  result  of  the  processing  on  the  VICOM  was  a  sparse  range  image  which 
had  range  values  in  pixels  identified  as  being  edges  in  the  intensity  image  and  zeros 
in  all  other  pixels.  The  image  was  transferred  to  a  VAX  where  it  was  searched  for 
non-zero  values.  When  a  non-zero  value  was  found  the  immediate  8-connected 
neighborhood  was  searched  for  non-zero  values,  the  search  continuing  from  pixel  to 
pixel  until  no  neighboring  non-zero  value  was  found.  This  algorithm  was 
implemented  to  track  a  line  in  both  directions  from  the  initial  point  and  to  cross 
breaks  one  pixel  wide.  Lists  of  points  longer  than  120  pixels  were  considered  to  be 
important  and  were  written  out  with  each  pixel  represented  by  three  numbers:  the 
row,  the  column,  and  the  value  (i.e.  the  range  in  machine  units).  Lists  of  points 
which  represented  curves  not  presently  of  interest  were  later  edited  out  by  hand. 

Smoothing  of  these  sets  of  points  was  done  using  the  algorithm  described  in 
[SchSh85].  The  procedure  to  smooth  a  curve  in  a  plane  can  be  thought  of  as 
extending  a  narrow  band  c  wide  about  the  curve,  considering  the  curve  to  be  a 
string  and  pulling  the  string  tight  within  the  constraints  of  the  narrow  band.  The 
smoothed  curve  can  be  described  by  the  list  of  resulting  corner  points.    The  3-D 
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smoothing  was  implemented  as  two  successive  2-D  smoothing  operations,  the  first  in 
the  image  plane  and  the  second  in  the  folded  plane  perpendicular  to  the  image  plane 
and  intersecting  it  in  the  piecewise  linear  arc  which  was  the  result  of  the  first 
smoothing.  The  c  used  was  1  pixel  in  the  512x460  image  plane  and  1  part  in  512  of 
the  range  values.  The  final  set  of  points  defined  the  ends  of  line  segments  which 
composed  the  piecewise  linear  approximating  curve  used  in  the  matching.  The 
smoothed  points  were  converted  to  points  in  real  space  using  the  range  value  as  the 
z  coordinate  and  the  pixel  location  with  the  camera  fields  of  view  to  generate  x  and 
y  values.  The  final  form  of  the  curve  for  a  figure  was  a  list  of  coordinates  in  real 
space  scaled  so  distances  were  in  centimeters. 

5.3.    Results  of  Experiments  with  Matching 

Input  to  the  matching  program  was  two  sequences  of  x,y,z  coordinates 
describing  piecewise  linear  curves  in  three  space  and  a  sampling  parameter  which 
was  the  interval  along  the  curves  at  which  test  points  were  selected.  Primary  output 
of  the  program  was  the  optimum  least  squares  difference  between  the  two  sets  of 
test  points  derived  from  the  curves.  In  the  following  paragraphs  we  describe  some 
experiments  to  determine  first  what  effects  the  choice  of  sampling  parameter  had, 
and  second,  how  well  the  matching  program  distinguished  among  objects,  some  of 
them  quite  similar,  on  the  basis  of  measurements  provided  by  the  range  sensor. 

Any  choice  of  sampling  parameter  is  a  compromise  between  a  small  value  that 
provides  better  representation  of  the  objects  and  a  large  value  that  allows  faster 
processing  because  fewer  points  are  involved.  To  investigate  the  quality  of 
matching  as  a  function  of  the  sampling  interval  six  sets  of  data  were  made  of  one 
figure  (the  large  balloon)  in  many  positions,  including  some  expected  to  produce 
poor  quality  data.  One  of  the  data  sets  was  designated  the  test  curve  and  was 
matched  against  each  of  the  others  using  many  different  sampling  intervals;  the  least 
square  differences  returned  by  the  matching  program  are  reproduced  in  Table  5.2. 
The  test  curve  was  not  an  ideal  model,  and  the  data  for  it  was  gathered  in  exactly 
the  same  way  as  for  objl  through  obj5. 
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interval 

objl 

obj2 

obj3 

obj4 

obj5 

1.2 

13.3 

36.6 

13.3 

16.7 

19.6 

.61 

1.98 

5.25 

5.24 

6.85 

11.29 

.60 

3.06 

7.09 

8.42 

8.57 

10.81 

.59 

1.92 

5.03 

4.94 

6.66 

10.75 

.31 

1.31 

2.80 

3.18 

5.78 

7.49 

.30 

1.30 

2.48 

2.76 

5.87 

7.43 

.29 

1.27 

2.17 

2.27 

5.69 

7.27 

.15 

0.94 

1.95 

1.71 

5.43 

6.72 

.10 

0.87 

1.64 

1.61 

5.31 

6.51 

Table  5.2  Effect  of  Sampling  Interval 

The  values  in  Table  5.2  are  the  observed  least  square  differences  multiplied  by 
100  to  provide  numbers  convenient  to  reproduce  and  work  with;  all  'difference' 
values  in  this  paper  implicitly  contain  this  factor  of  100.  The  most  noticeable  trend 
in  the  table  is  that  the  returned  difference  is  smaller  as  the  sampling  interval  is 
made  smaller.  This  is  to  be  expected,  because  a  better  fit  can  be  found  with  more 
points  (possible  solutions)  to  choose  from  and  because  the  distance  between  test 
points  on  curves  that  match  very  well  must  necessarily  decrease.  However,  as  the 
sampling  interval  was  made  smaller  the  difference  for  curves  which  do  not  match 
well  did  not  decrease,  or  decreased  only  very  slowly;  each  object  in  Table  5.2  shows 
this  effect.  (For  sampling  intervals  as  small  as  0.02  the  differences  for  these  objects 
were  within  0.1  of  the  value  for  the  smallest  interval  in  the  table.)  When  the 
sampling  interval  was  too  large  erratic  behavior  was  observed,  as  for  instance  in 
obj2  with  interval  1.2,  and  in  objl  with  interval  0.60.  Even  the  relative  ranking  of 
matches  changed,  as  in  obj2  and  obj3  with  interval  0.59. 

On  the  basis  of  measurements  such  as  these  we  chose  to  use  a  sampling 
interval  of  0.3  for  all  work  with  curves  from  the  flower  pot  figures  since  this  was 
the  largest  interval  which  produced  consistent  results.  This  interval  was  about  2-3% 
of  the  length  of  the  curves  used  in  these  experiments.  With  this  interval 
approximately  75  seconds  per  match  was  required  on  a  normally  loaded  VAX  750. 

With  a  sampling  parameter  of  0.3  a  difference  value  between  1  and  5  signifies 
a  very  good  match  between  curves,  a  difference  less  than  10  suggests  good 
correlation  but  slight  deformation,  and  a  difference  greater  than  20  indicates 
significant  mismatch.  Differences  greater  than  100  have  been  observed  between 
some  of  these  curves,  which  means  that  the  average  distance  between  the  curves 
could  not  be  made  smaller  than  1  cm.  Note  that  even  when  identical  curves  are 
matched  the  least  squares  difference  may  be  greater  than  zero,  since  if  the  sampling 
of  one  closed  curve  starts  at  a  different  location  than  on  the  other  curve  the  test 
points  generally  do  not  coincide.  A  typical  value  of  0.4  was  found  for  the 
difference  between  identical  data  sets  sampled    from  different  starting  points  with 
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an  interval  of  0.3. 

A  crude  measure  of  the  performance  of  the  data  gathering  and  matching 
protocol  is  simply  to  match  two  sets  of  curves  digitized  from  the  general  vocabulary 
of  figures.  The  differences,  multiplied  by  100  as  usual,  are  given  in  Table  5.3, 
where  a  small  difference  indicates  a  good  match.  Each  row  of  the  table  represents 
the  match  values  of  one  old  curve  with  all  of  the  new  curves,  and  each  column  one 
new  curve  with  all  the  old  ones.  The  two  sets  of  data  were  generated  more  than  a 
month  apart,  using  the  same  calibration.  The  figures  were  imaged  in  different 
locations  and  orientations  in  the  workspace.  Orthographic  projections  of  the 
juxtaposed  peanut  and  pansy  curves  before  and  after  matching  are  shown  in  fig.  5.5. 


bean 

palette 

pansy 

peanut 

sail 

sundrop 

BEAN 

20.3 

97.3 

95.0 

48.9 

36.8 

78.1 

PALETTE 

102.0 

7.8 

40.6 

116.3 

118.8 

30.5 

PANSY 

93.1 

30.3 

6.0 

117.6 

100.8 

28.7 

PEANUT 

48.2 

101.0 

145.7 

3.8 

77.5 

106.3 

SAIL 

19.8 

82.4 

87.3 

71.7 

4.1 

81.7 

SUNDROP 

73.5 

16.9 

31.6 

93.5 

95.2 

4.2 

Table  5.3   Matching  Differences  for  General  Figures 

In  almost  every  case  the  correct  figure  was  distinguished  from  the  other 
figures.  The  only  exception  was  the  bean  figure,  and  examination  of  the  data 
showed  that  the  early  curve  was  deformed  by  a  sharp  spike  in  one  area.  Spikes  of 
this  sort  were  found  to  be  caused  by  pinholes  in  one  of  the  filters,  and  some  care 
was  taken  in  later  work  to  avoid  having  a  boundary  sit  in  these  sharp  local 
variations  in  intensity. 

Since  the  matching  algorithm  was  designed  to  discover  which  of  a  set  of  model 
objects  an  observed  object  best  matches  it  did  not  even  attempt  to  match  observed 
objects  with  models  which  were  smaller.  Table  5.3  is  a  composite  of  the  results 
obtained  from  matching  one  set  (e.g.  bean)  against  the  other  (e.g.  BEAN)  and  then 
vice  versa.  In  some  cases  objects  were  sufficiently  close  in  size  that  they  were 
matched  both  ways.  Usually  the  two  difference  values  were  within  1  of  each  other, 
but  larger  variations  were  observed  with  mismatching  figures,  such  as  sundrop  with 
sail. 

The  most  startling  difference  between  the  double  matching  results  was  in  the 
correct  match  of  pansy  with  PANSY;  the  value  in  one  direction  was  6.0  while  the 
match  in  the  other  direction  was  11.7.  While  this  is  not  clearly  a  mismatch  (the 
difference  is  still  low  compared  to  matches  with  other  figures),  it  is  not  a  good 
match.  The  deeply  reentrant  corners  of  the  pansy  figure  appear  to  cause  the 
problem,  since  the  location  of  the  sampled  test  points  in  space  is  more  sensitive  to 
the  sampling  starting  point  than  in  smoother  figures.  Matching  in  the  'bad'  direction 
using  a  sampling  parameter  of  0. 1  instead  of  0.3  resulted  in  a  difference  of  5. 1 ,  with 
only  very  slight  changes  in  the  matching  position.  Sundrop  also  has  moderately 
sharp  indentations,  and  the  difference  for  its  reverse  match  was  6.5,  different  from 
4.2. 
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To  investigate  the  ability  of  the  matching  algorithm  to  distinguish  among 
figures  of  similar  shape  but  different  size  a  number  of  images  were  made  of  the 
balloon  figures.  The  figures  were  imaged  in  different  orientations  and  positions 
spanning  20  cm  in  range  about  a  point  70  cm  from  the  camera.  Representative 
results  of  the  matching  of  every  image  with  every  other  are  reproduced  in  Table 
5.4.  The  low  matching  values  within  each  figure  and  the  large  values  between  the 
figures  demonstrate  that  the  matching  discriminated  between  objects  using 
geometric  data  provided  by  the  range  sensor.  It  should  be  borne  in  mind  that  the 
size  of  the  image  of  a  figure  on  the  retina  of  the  camera  changes  inversely  with 
distance  of  the  figure  from  the  camera.  Thus  the  small  balloon  at  64  cm  will  have 
the  same  size  (measured  in  pixels)  as  the  large  balloon  at  76  cm. 


bala 

balb 

bale 

2    3 

4 

1 

2 

3 

4 

5 

1 

2 

3 

4 

balal   1.7   3.7 

4.1 

13 

8.7 

14 

14 

9.5 

45 

42 

57 

41 

2 

3.6 

4.5 

12 

11 

9.4 

15 

8.2 

50 

46 

66 

47 

3 

4.1 

8.5 

5.0 

8.0 

9.3 

5.2 

40 

38 

53 

38 

4 

7.4 

6.9 

8.6 

12 

5.4 

43 

38 

58 

41 

balbl 

2.7 

2.5 

2.4 

1.4 

23 

19 

34 

21 

2 

4.1 

3.9 

1.4 

29 

26 

38 

27 

3 

4.3 

3.2 

30 

24 

47 

29 

4 

4.1 

18 

15 

28 

17 

5 

27 

23 

40 

25 

balcl 

1.3 

3.2 

2.1 

2 

5.9 

2.5 

3 

6.1 

Table  5.4     Matching  Differences  for  Balloon  Figures 

Examination  of  the  matching  results  has  shown  no  matching  error  traceable  to 
rotation  of  the  figures.  There  is,  however,  evidence  of  a  small  change  in  the 
apparent  size  of  the  figures  in  the  matching  of  the  balb  curves  with  the  larger  and 
smaller  bale  and  bala  figures.  The  curve  balb4  was  imaged  with  the  figure  at  about 
60  cm  and  balb5  was  imaged  at  about  80  cm.  Balb5  matches  somewhat  better  with 
the  small  figure  bala  than  balb4  and  balb4  matches  the  large  figure  better  than 
balb5;  from  this  it  can  be  inferred  that  the  figures  appear  smaller  as  they  are  moved 
away  from  the  camera.  The  change  was  probably  less  than  2%  in  the  space 
examined,  and  change  may  be  caused  by  a  small  error  in  the  measurement  of  the 
camera  field  of  view.  It  could  also  result  from  smoothing  being  done  at  the  pixel 
level  without  regard  for  the  fact  that  the  amount  of  the  figure  contained  in  a  pixel 
varies  with  distance  from  the  camera. 

The  last  experiment  matched  figures  identical  in  size  but  of  different  shape. 
Three  rectangles  were  cut  to  be  identical  in  size,  and  then  were  affixed  to  pots  in 
such  a  manner  that  they  assumed  different  curvatures.  Recv  was  closest  to  a  plane 
figure,  having  its  long  sides  upright  on  the  pot  in  a  normal  standing  position).  Reel 
was  slightly  more  curved,  being  glued  to  the  large  pot  close  to  the  upper  rim  with  its 
long  sides  going  around  the  pot.   Recs  was  the  most  curved,  being  mounted  with  its 
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long  sides  extending  around  the  small  pot  close  to  the  bottom. 


recv 
12             3             4 

reel 
12             3             4 

recs 
12            3            4 

recvl 
2 
3 

4 

3.5 
3.9 

3.0        4.0 
4.0         4.0 

4.5 

6.5         5.8         8.8       15 
5.1         5.7         8.6       15 
5.5         5.7         7.8       13 

5.4         6.2         7.7       14 

16 

11 

recll 
2 
3 

4 

; 

5.4 
5.6 
7.3 

3.1         2.4         6.9 
3.2          -            3.3         7.0 
2.4         3.2          -            4.3 

rccsl 
2 
3 
4 

13            9.8       15          11 
19          17          18          16 
16          14          15          13 
12          11          11            9.8 

11          11          13          16 
11           13           10             9.5 
11          10            9.6         9.8 
8.4         9.4         7.7         8.1 

8.8       5 

3.7 
5.2 

2        5.2 
5.3 
3.6 

Table  5.5    Matching  Differences  for  Rectangle  Figures 

Table  5.5  reproduces  the  results  of  matching  all  rectangle  curves  with  all 
others.  As  before,  the  figures  were  placed  in  different  locations  and  orientations  in 
the  workspace  for  the  different  data  sets.  Even  though  the  figures  are  similar  the 
program  invariably  finds  a  best  match  for  a  curve  with  another  curve  made  from  the 
same  figure.  In  addition  the  reported  differences  correlate  well  with  an  intuitive 
idea  of  how  different  the  figures  are.  In  this  table  the  dot  indicates  that  the  model 
was  found  to  be  too  small;  only  those  figures  having  exactly  the  same  number  of  test 
points  matched  both  ways,  as  for  example  recvl  and  recv4,  or  recv2  with  recs2. 
The  dash  simply  fills  space,  since  no  object  is  matched  against  itself. 

6.    Summary 

We  have  presented  experiments  in  which  objects  in  three-space  have  been 
successfully  identified  matching  3-D  curves  extracted  from  their  surfaces.  The 
experiments  use  data  extracted  from  registered  range  and  intensity  images  which 
were  generated  by  recently  developed  range  sensor,  a  method  for  extracting  3-D 
characteristic  curves  from  these  images,  an  extension  of  a  2-D  curve  smoothing 
procedure  to  3-D  curves,  and  an  algorithm  which  matches  general  3-D  curves  by 
least  squares  fit. 

The  algorithm  runs  in  0(m  log  m)  time,  where  m  is  proportional  to  the 
combined  length  of  the  curves.  The  algorithm  can  be  used  to  match  complete  or 
partial  curves  against  a  model,  where  the  model  is  represented  also  as  a  curve.  The 
strengths  of  the  algorithm  are  its  generality  and  efficiency;  its  weakness  is  its 
sensitivity  to  distortions  of  the  arclength  parametrization  by  noise.  In  the 
experiments  of  section  5  an  observed  object  was  matched  against  a  library  of 
previously  observed  objects  by  performing  a  match  between  the  observed  object  and 
each  model  sequentially.  This  method  of  discovering  the  best  matching  model  can 
easily  be  run  on  many  processors  in  parallel. 
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An  important  bridge  between  the  lists  of  points  provided  by  the  range  sensor 
and  the  piecewise  linear  curves  used  by  the  matching  algorithm  is  the  smoothing 
algorithm.  Range  data  contains  distortions  in  the  range  values  due  to  noise  from 
many  sources  (see  section  4.3)  and  also  distortions  due  simply  Lo  recording  a  curve 
on  a  rectangular  grid.  This  routine  is  designed  to  smooth  a  curve  with  no 
assumptions  about  the  shape  of  the  curve.  An  assumption  underlying  the  use  of  this 
routine  is  that  points  do  not  deviate  from  their  proper  value  by  more  than  a  certain 
maximum  value;  this  would  seem  to  make  it  perform  poorly  with  data  having 
normal  error  distribution.  However,  in  practice  the  routine  behaves  quite  well. 
The  smoothing  parameter  (c)  used  in  the  experiments  reported  here  was  about  1.4 
mm. 

The  range  sensor  used  in  these  experiments  makes  a  range  image  with  an 
average  uncertainty  of  about  2  mm  in  a  working  area  86  cm  deep  centered  about  41 
cm  from  the  camera  in  40  seconds.  This  uncertainty  could  be  reduced  by  averaging 
intensity  digitizations,  since  much  of  the  uncertainty  comes  from  random  noise 
associated  with  the  camera.  The  most  troublesome  deviations  in  a  range  image  are 
those  caused  by  pinholes  in  the  filters  or  dirt  on  the  filters;  dirt  on  other  optical 
components  is  less  troublesome  because  it  is  not  focused  in  the  work  area  of  the 
sensor. 

The  strengths  of  this  sensor  are  its  speed  and  the  image  format  of  the  output, 
since  many  low  level  procedures  are  easily  implemented  to  work  on  images.  In 
addition  the  sensor  is  relatively  cheap  (assuming  one  has  z  suitable  image 
processing  machine),  easy  to  use  and  safe.  It  is  quickly  calibrated  and  is  not  overly 
sensitive  to  small  changes  in  camera  or  projector  focus.  The  basic  sensor  can  be 
configured  in  many  ways;  for  example,  if  the  camera  and  projector  are  fixed  to  a 
board  the  sensor  could  be  calibrated  on  a  table  top  then  placed  above  and  to  the  side 
of  a  robot  installation,  with  the  board  slanted  to  look  down.  (The  slide  projector 
functions  well  at  angles  as  much  as  45°  from  the  horizontal.) 

There  is  of  course  further  developmental  work  to  be  done  on  the  range  sensor. 
The  addition  of  a  second  projector  will  reduce  the  portion  of  the  image  which  is  in 
shadow  or  poorly  illuminated;  note  that  there  should  be  no  problem  in  combining 
range  images  since  all  images  are  made  by  a  single  camera.  The  use  of  filters  which 
have  multiple  gradients,  such  as  a  sawtooth  function  for  the  transmissivity,  would 
improve  the  range  resolution.  A  projector  could  be  built  to  change  the  illumination 
more  quickly  so  that  it  would  take  only  a  second  to  make  the  24  images  used  for  the 
range  images  of  these  experiments,  and  special  hardware  could  be  built  to  perform 
the  computations  in  something  like  a  pipeline  mode. 

Perhaps  the  biggest  contribution  of  the  range  sensor  is  that  it  makes  available 
good  quality  registered  range  and  intensity  images.  In  the  experiments  described 
above  only  the  simplest  use  was  made  of  this  combination  of  images.  One  can 
envision  low  level  processes  which  identify  different  types  of  lines,  e.g.  pure 
discontinuities  in  the  range  image,  occluding  edges  (identified  by  large  gradients  in 
the  range  image  near  a  discontinuity),  or  changes  in  local  slope  or  curvature;  some 
curves  may  even  be  developed  by  combining  intensity  images  from  the  two 
projectors  mentioned  above.  Solid  objects  in  3-D  would  then  be  described  as  sets  of 
curves,  each  curve  having  a  character  or  type  as  well  as  a  location  in  space  (or 
perhaps  a  location  relative  to  the  other  curves  of  its  set).    Object  identification 
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might  then   involve   identifying  curves  and  verifying  proper  correspondence   with 
other  curves  of  the  set. 
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Fig.  1.1    Sketch  of  an  object.   Lines  represent  projections  of  curves  which 
are  significant  to  shape. 
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Fig.  4.1  2-D  version  of  a  depth  sensor.  Work  area  of  the  sensor  is  the 
region  both  illuminated  by  the  projector  and  viewed  by  the  camera. 
Intensity  of  the  light  is  varied  across  the  beam  by  a  filter. 
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Fig.  4.2    Geometrical  relations  and  quantities  used  in  the  derivation  of 
equation  4.5. 


Fig.  5.1  Set  of  general  figures  used  in  the  matching  experiments.  The 
top  row,  from  left  to  right,  are  bean,  palette,  pansy;  the  bottom  row  are 
peanut,  sail,  and  sundrop.  The  figures  are  seen  as  projected  onto  the 
camera  retina,  the  boxes  outlining  about  one  tenth  of  the  projected  area. 


Fig.  5.2  Model  shape  for  each  of  the  balloon  figures,  which  were  of 
slightly  different  sizes  (see  text).  The  figure  is  composed  of  two  circles, 
one  having  half  the  radius  of  the  other  and  drawn  about  a  point  on  the 
circumference  of  the  larger,  and  their  common  tangents. 


Fig.  5.3  Curves  representing  digitizations  of  the  balloon  figures,  as 
projected  onto  the  camera  retina.  Scale  is  the  same  for  each;  the 
variation  in  size  and  shape  is  caused  by  the  objects  standing  in  different 
regions  of  the  workspace.  Left  column  is  balal-4,  center  column  is 
balbl-4,  right  column  is  balcl-4,  in  order  top  to  bottom. 
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Fig.  5.4  Curves  representing  digitizations  of  the  rectangle  figures.  The 
upper  row  has  the  figures  recs,  reel  and  recv  from  left  to  right  as 
projected  onto  the  camera  retina.  The  same  figures  are  shown  in  the 
second  row  after  having  been  smoothed.  The  rectangles  appear  to  be 
curved  because  each  is  affixed  to  the  surface  of  a  flower  pot. 


Fig  5.5  Results  of  matching  digitizations  of  the  general  figures  peanut 
(left)  and  pansy  (right).  These  curves  are  shown  in  orthographic 
projection,  and  the  overlaid  curves  which  show  the  result  of  the  matching 
process  (third  row)  are  not  drawn  to  the  same  scale  as  the  model  curves. 
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